---
subtitle: Guides you through the process of creating and managing datasets
---

Datasets can be used to track test cases you would like to evaluate your LLM on. Each dataset is made up of a dictionary
with any key value pairs. When getting started, we recommend having an `input` and optional `expected_output` fields for
example. These datasets can be created from:

- Python SDK: You can use the Python SDK to create a dataset and add items to it.
- TypeScript SDK: You can use the TypeScript SDK to create a dataset and add items to it.
- Traces table: You can add existing logged traces (from a production application for example) to a dataset.
- The Opik UI: You can manually create a dataset and add items to it.

Once a dataset has been created, you can run Experiments on it. Each Experiment will evaluate an LLM application based
on the test cases in the dataset using an evaluation metric and report the results back to the dataset.

## Create a dataset via the UI

The simplest and fastest way to create a dataset is directly in the Opik UI.
This is ideal for quickly bootstrapping datasets from CSV files without needing to write any code.

Steps:

1. Navigate to **Evaluation > Datasets** in the Opik UI.
2. Click **Create new dataset**.
3. In the pop-up modal:
   - Provide a name and an optional description
   - Optionally, upload a CSV file with your data
4. Click **Create dataset**.

<Frame>
  <img src="/img/evaluation/create_dataset.png" />
</Frame>

If you need to create a dataset with more than 1,000 rows, you [can use the SDK](/evaluation/manage_datasets#creating-a-dataset-using-the-sdk).

<Tip>
  The UI dataset creation has some limitations:
    * File size is limited to 1,000 rows via the UI.
    * No support for nested JSON structures in the CSV itself.

For datasets requiring rich metadata, complex schemas, or programmatic control, use the SDK instead (see the next section).

</Tip>

## Adding traces to a dataset

One of the most powerful ways to build evaluation datasets is by converting production traces into dataset items. This allows you to leverage real-world interactions from your LLM application to create test cases for evaluation.

### Adding traces via the UI

To add traces to a dataset from the Opik UI:

1. Navigate to the traces page
2. Select one or more traces you want to add to a dataset
3. Click the **Add to dataset** button in the toolbar
4. In the dialog that appears:
   - Select an existing dataset or create a new one
   - Choose which trace metadata to include:
     - **Nested spans**: Include all child spans within the trace
     - **Tags**: Include trace tags
     - **Feedback scores**: Include any feedback scores attached to the trace
     - **Comments**: Include comments added to the trace
     - **Usage metrics**: Include token usage and cost information
     - **Metadata**: Include custom metadata fields
5. Click on the dataset name to add the selected traces

<Frame>
  <img src="/img/evaluation/add_traces_to_dataset.png" alt="Add traces to dataset modal" />
</Frame>

<Tip>
  By default, all metadata options are enabled. You can uncheck any options you don't need. The trace's input and output are always included.
</Tip>

### What gets added to the dataset

When you add a trace to a dataset, the following structure is created:

- **input**: The trace's input data
- **expected_output**: The trace's output data (stored as `expected_output` for evaluation purposes)
- **spans** (optional): Array of nested spans with their inputs, outputs, and metadata
- **tags** (optional): Array of tags associated with the trace
- **feedback_scores** (optional): Array of feedback scores with name, value, and source
- **comments** (optional): Array of comments with text and ID
- **usage** (optional): Token usage and cost information
- **metadata** (optional): Custom metadata fields

This rich structure allows you to:
- Evaluate complex multi-step workflows by including nested spans
- Filter and analyze based on tags and metadata
- Use existing feedback scores as ground truth for evaluation
- Preserve context through comments and annotations

## Creating a dataset using the SDK

You can create a dataset and log items to it using the `get_or_create_dataset` method:

<CodeBlocks>
```typescript title="TypeScript SDK" language="typescript"
import { Opik } from "opik";

// Create a dataset
const client = new Opik();
const dataset = await client.getOrCreateDataset("My dataset");
```

```python title="Python SDK" language="python"
from opik import Opik

# Create a dataset
client = Opik()
dataset = client.get_or_create_dataset(name="My dataset")
```

</CodeBlocks>

If a dataset with the given name already exists, the existing dataset will be returned.

### Insert items

#### Inserting dictionary items

You can insert items to a dataset using the `insert` method:

<CodeBlocks>
```typescript title="TypeScript" language="typescript"
import { Opik } from "opik";
const client = new Opik();
const dataset = await client.getOrCreateDataset("My dataset");

dataset.insert([
  { user_question: "Hello, world!", expected_output: { assistant_answer: "Hello, world!" } },
  { user_question: "What is the capital of France?", expected_output: { assistant_answer: "Paris" } },
]);
```

```python title="Python" language="python"
import opik

# Get or create a dataset
client = opik.Opik()
dataset = client.get_or_create_dataset(name="My dataset")

# Add dataset items to it
dataset.insert([
    {"user_question": "Hello, world!", "expected_output": {"assistant_answer": "Hello, world!"}},
    {"user_question": "What is the capital of France?", "expected_output": {"assistant_answer": "Paris"}},
])
```

</CodeBlocks>

<Tip>
  Opik automatically deduplicates items that are inserted into a dataset when using the Python SDK. This means that you
  can insert the same item multiple times without duplicating it in the dataset. This combined with the `get or create
  dataset` methods means that you can use the SDK to manage your datasets in a "fire and forget" manner.
</Tip>

Once the items have been inserted, you can view them in the Opik UI:

<Frame>
  <img src="/img/evaluation/dataset_items_page.png" />
</Frame>

#### Inserting items from a JSONL file

You can also insert items from a JSONL file:

<CodeBlocks>
  ```python title="Python" language="python"
  import opik

  client = opik.Opik()
  dataset = client.get_or_create_dataset(name="My dataset")

dataset.read_jsonl_from_file("path/to/file.jsonl")

```
</CodeBlocks>

#### Inserting items from a Pandas DataFrame

You can also insert items from a Pandas DataFrame:

<CodeBlocks>
```python title="Python" language="python"
import opik

client = opik.Opik()
dataset = client.get_or_create_dataset(name="My dataset")

dataset.insert_from_pandas(dataframe=df)

# You can also specify an optional keys_mapping parameter
dataset.insert_from_pandas(dataframe=df, keys_mapping={"Expected output": "expected_output"})
```

</CodeBlocks>

### Deleting items

You can delete items in a dataset by using the `delete` method:

<CodeBlocks>
  ```typescript title="TypeScript" language="typescript"
  import { Opik } from "opik";

  // Get or create a dataset
  client = new Opik();
  dataset = await client.getDataset("My dataset")

  await dataset.delete(["123", "456"])

  // Or to delete all items
  await dataset.clear()
  ```

  ```python title="Python" language="python"
  from opik import Opik

  # Get or create a dataset
  client = Opik()
  dataset = client.get_dataset(name="My dataset")

  dataset.delete(items_ids=["123", "456"])

  # Or to delete all items
  dataset.clear()
  ```
</CodeBlocks>

## Downloading a dataset from Opik

You can download a dataset from Opik using the `get_dataset` method:

<CodeBlocks>
  ```typescript title="TypeScript" language="typescript"
  import { Opik } from "opik";

  const client = new Opik();
  const dataset = await client.getDataset("My dataset");

  const items = await dataset.getItems();
  console.log(items);
  ```

  ```python title="Python" language="python"
  from opik import Opik

  client = Opik()
  dataset = client.get_dataset(name="My dataset")

  # Get items as list of DatasetItem objects
  items = dataset.get_items()

  # Convert to a Pandas DataFrame
  dataset.to_pandas()

  # Convert to a JSON array
  dataset.to_json()
  ```
</CodeBlocks>

## Expanding a dataset with AI

Dataset expansion allows you to use AI to generate additional synthetic samples based on your existing dataset. This is particularly useful when you have a small dataset and want to create more diverse test cases to improve your evaluation coverage.

The AI analyzes the patterns in your existing data and generates new samples that follow similar structures while introducing variations. This helps you:

- **Increase dataset size** for more comprehensive evaluation
- **Create edge cases** and variations you might not have considered
- **Improve model robustness** by testing against diverse inputs
- **Scale your evaluation** without manual data creation

### How to expand a dataset

To expand a dataset with AI:

1. **Navigate to your dataset** in the Opik UI (Evaluation > Datasets > [Your Dataset])
2. **Click the "Expand with AI" button** in the dataset view
3. **Configure the expansion settings**:
   - **Model**: Choose the LLM model to use for generation (supports GPT-4, GPT-5, Claude, and other models)
   - **Sample Count**: Specify how many new samples to generate (1-100)
   - **Preserve Fields**: Select which fields from your original data to keep unchanged
   - **Variation Instructions**: Provide specific guidance on how to vary the data (e.g., "Create variations that test edge cases" or "Generate examples with different complexity levels")
   - **Custom Prompt**: Optionally provide a custom prompt template instead of the auto-generated one
4. **Start the expansion** - The AI will analyze your data and generate new samples
5. **Review the results** - New samples will be added to your dataset and can be reviewed, edited, or removed as needed

<Frame>
  <img src="/img/evaluation/dataset_expansion_modal.png" />
</Frame>

### Configuration options

**Sample Count**: Start with a smaller number (10-20) to review the quality before generating larger batches.

**Preserve Fields**: Use this to maintain consistency in certain fields while allowing variation in others. For example, preserve the `category` field while varying the `input` and `expected_output`.

**Variation Instructions**: Provide specific guidance such as:

- "Create variations with different difficulty levels"
- "Generate edge cases and error scenarios"
- "Add examples with different input formats"
- "Include multilingual variations"

### Best practices

- **Start small**: Generate 10-20 samples first to evaluate quality before scaling up
- **Review generated content**: Always review AI-generated samples for accuracy and relevance
- **Use variation instructions**: Provide clear guidance on the type of variations you want
- **Preserve key fields**: Use field preservation to maintain important categorizations or metadata
- **Iterate and refine**: Use the custom prompt option to fine-tune generation for your specific needs

<Tip>
  Dataset expansion works best when you have at least 5-10 high-quality examples in your original dataset. The AI uses
  these examples to understand the patterns and generate similar but varied content.
</Tip>

## Managing dataset item tags

Tags are a powerful way to organize, categorize, and filter your dataset items. You can use tags to:

- **Categorize test cases** by type, difficulty, or domain (e.g., `edge-case`, `production`, `multilingual`)
- **Track data sources** where items originated from (e.g., `user-feedback`, `synthetic`, `real-world`)
- **Mark review status** during dataset curation (e.g., `needs-review`, `validated`, `archived`)
- **Filter for evaluation** to run experiments on specific subsets of your data
- **Organize workflows** by marking items for different stages or teams

Each dataset item can have multiple tags.

### Adding tags to dataset items

#### Adding tags to individual items

To add tags to a single dataset item:

1. **Navigate to your dataset** in the Opik UI (Evaluation > Datasets > [Your Dataset])
2. **Click on any dataset item** to open the details panel
3. **In the Tags section**, click the **"+" button**
4. **Type the tag name** and press Enter
5. The tag will be immediately added and saved

You can remove tags by clicking the **"×" icon** next to any tag in the details panel.

#### Adding tags to multiple items (batch operation)

To add the same tag to multiple dataset items at once:

1. **Navigate to your dataset** in the Opik UI
2. **Select multiple items** by clicking the checkboxes next to each item
3. **Click the "Add tags" button** in the toolbar (visible when items are selected)
4. **Enter the tag name** in the dialog that appears
5. **Click "Add tag"** to apply the tag to all selected items

This is particularly useful when you want to categorize a group of related test cases or mark items from the same data source.

<Tip>
  Tags are case-sensitive and support alphanumeric characters, hyphens, and underscores. Choose consistent naming conventions for your tags to make filtering easier.
</Tip>

### Filtering dataset items by tags

Once you've tagged your dataset items, you can filter them to work with specific subsets:

1. **Navigate to your dataset** in the Opik UI
2. **Click the "Filters" button** next to the search bar
3. **Select "Tags" from the Column dropdown**
4. **Choose "contains" as the operator**
5. **Enter the tag name** you want to filter by
6. **Close the dialog** to apply the filter

The dataset items table will update to show only items matching your filter criteria. You can:

- **View filtered items** to focus on specific categories
- **Run experiments** on filtered subsets by using the filtered view
- **Export filtered data** for specific test case groups
- **Combine with other filters** to create complex queries

The filter is saved in the URL, so you can bookmark or share specific filtered views of your dataset.
